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ABSTRACT. Define a truncation r t (p) of a polynomial p in {xj , X2,X3, . . .} as the polyno- 
mial with all but the first t variables set to zero. In certain good cases, the truncation of 
a Schubert or Grothendieck polynomial may again be a Schubert or Grothendieck poly- 
nomial. We use this phenomenon to give subtraction-free formulae for certain Schubert 
structure constants in K(Flags(C n )), in particular generalizing those from [Kogan, '00] in 
which only cohomology was treated, and from [Buch, '02] on the Grassmannian case. The 
terms in the answer are computed using "marching" operations on permutation diagrams. 

1. Introduction 

Let Flags(C n ) denote the variety of complete flags in C n . To each permutation n in 
the symmetric group S n , there is an associated Schubert variety C Flags(C n ). The 
classes of the Schubert structure sheaves [0 Xn ] form an additive Z-linear basis of the K- 
theory (Grothendieck) ring K(Flags(C n )) of algebraic vector bundles over Flags(C n ). The 
Schubert structure constants are the integers defined by 

(i) io x j-[o Xp ] = ^ci P [o x j. 

It is known |Br02| that (-1 )«M+«(pMWC£ p > 0, where t{<x) is the minimum I such that a 
is expressible as a product of I simple transpositions Si = t^t+i. In the cohomology case, 
i.e., when +£(p) = £(7t), these are the structure constants for the analogous expansion 
of the product of Schubert classes [XJ ■ [X p ] in the cohomology ring H*(Flags(C n )), and 
count the number of points in the intersection of general triple translates of X a , X p and 
X Wo7T (where w denotes the longest permutation in S n ). The expansion ^ behaves well 
with respect to the inclusion S n ^4 S n+ i. In particular, for any two permutations cr, p, and 
n sufficiently large, © stabilizes. Therefore, it will be unambiguous (and convenient) to 
call (cr, p, 7i) G a Schubert problem, where Soo = Un>i ^ Ti- 
lt is a famous open problem to give a general subtraction-free combinatorial formula 
applicable to any Schubert problem. The analogous problem for Grassmannians is solved 
in the cohomology case by the Littlewood-Richardson rule, and more recently in K-theory 
by A. Buch |Bu02a[. A solution for the flag variety would provide an important general- 
ization of the Littlewood-Richardson rule. However, the known generalized Littlewood- 
Richardson rules handle only limited cases of the Schubert problems, with I Ko01llBu 02al 
representing the state of the art. 

Our main result is a subtraction-free combinatorial formula for the family we call trun- 
cation Schubert problems (defined below). This formula specializes to compute the K- 
theory generalizations of the numbers considered by M. Kogan |Ko01[ and the K-theory 

Date: Draft of February 1, 2008. 
AK was supported by an NSF grant. 



1 



Littlewood-Richardson coefficients of |Bu02a|. Actually, our main result gives formu- 
las for many other combinatorial numbers studied in connection to the Schubert calcu- 
lus IBu02allL01llBS9 8 |, formulas for Schubert and Grothendieck polynomials |LS82 , BJS93 . 
IFRMIFK961 . deg eneracy loci [ B F9j lBu02bl IF991 IBTTY021 BKTY03 1 and quantum Schu- 
bert polynomials IIFGF971 ICF991 IBKTYU2I , see, e.g., IBSY03II and the references therein. 
We find it interesting that our formula also applies to new cases of Schubert problems 
where neither class is a pullback from a Grassmannian. 

The fact that these numbers all arise from the single framework of problems isolated 
here suggests that their common combinatorial and geometric features ought to be better 
understood. 

On the combinatorial side, in Section 3, we present our formula in terms of simple 
"marching" moves of the diagram of a permutation. We would like to understand how, 
e.g., various combinatorial aspects of the classical Littlewood-Richardson coefficients 
might extend to this family of numbers. It would be interesting to understand the rela- 
tions between the formula given here and formulas for the aforementioned special cases, 
and other related formulas, e.g., llFLMILe03llLS0llPR^IS9^ . 

One feature of our proof is that it is both short and completely combinatorial. It is 
based on "truncation" techniques concerning Grothendieck polynomials |LS82| and in 
particular, the "transition" formula of A. Lascoux HL01I1 . These methods (at least in co- 
homology) can be considered classical in the subject. Indeed, in previous work |L01| 
(see also |LS85|), similar techniques were applied to give new formulas for the K-theory 
Littlewood-Richardson coefficients (after |Bu02a|). However, it is perhaps surprising that 
such methods are in fact applicable to more general Schubert calculus problems, and in 
particular, Kogan's Schubert problems. Our principal novelty of three simultaneous ob- 
servations is reflected respectively in the three equalities found in © from Section 3. 

Thus, since Kogan's Schubert problems form a special case of the truncation Schubert 
problems, our formula covers new cases of the Schubert problem in K-theory (and more- 
over, our proof makes transparent the role of Kogan's conditions). However, we empha- 
size that our formula handles new cases beyond that in |Ko01 1, even in cohomology. 

A further goal of this paper is to present the diagram marching moves. One reason 
to use such (recursive) combinatorics is that the moves have a natural geometric inter- 
pretation. In a sequel |KY04| to this paper, we interpret the moves in terms of Grobner 
degeneration of matrix Schubert varieties |F92| via diagonal term orders (in an important 
contrast to the anfz-diagonal term orders used in |KM01|). For example, in the cohomol- 
ogy case, our formula can be interpreted as counting certain components of a partially 
degenerated matrix Schubert variety. It would be interesting to understand what rela- 
tions exist between the formula presented here and the geometric Littlewood-Richardson 
rule of R. Vakil |V03|, which is also based on degeneration. 

Finally, one other advantage of the approach presented here is the possible extensions 
to other Schubert calculus settings, e.g., the cohomology /K-theory ring of flag varieties 
corresponding to the other classical Lie types BCD (work in progress with F. Sottile). 

We thank Frank Sottile and Alexander Woo for helpful discussions. 
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2. Diagram moves and the main result 



Let G(7t) denote the permutation matrix associated to n £ S n , and call the nonzero 
entries of G (7t) its dots. The diagram of a permutation n is the following subset of [n] x [n] : 

D(7t) :={(p,q), 1 <p,q < n, 7t(p) > q, 7t _1 (q) > p}. 

Equivalently, (p, q) £ D(7t) if (p, 7t _1 (q)) is an inversion of n; thus #D(7r) = t[n). Graphi- 
cally, D(7t) is obtained from G(7t) by drawing a "hook" consisting of lines going east and 
south from each dot. The diagram appears as a collection of "connected components" of 
squares not in the hook of any dot (see Example 1 below). 

Call the southernmost, then eastmost, box (I, m) £ D(7t) the maximal corner. Note that 
the maximal corner of n is in row I if and only if the last descent of n is in row I, i.e., the 
largest index I such that 7t(l) > 7t(l + 1 ). Call any dot that is maximally southeast with 
respect to the condition that it is northwest of (I, m) a pivot. There are no pivots if and 
only if the maximal corner is in the connected component of D(7t) attached to the top left 
corner of [n] x [n]. See Example 1 below. 

In the following definitions, it is convenient to describe the cohomology versions first 
before explaining their K-theory analogues. In the next section, we will connect what 
follows to the Grothendieck transition formula of IL01II . 

First, we describe the marching operation on D(7t). Suppose the maximal corner is 
at (I, m). If the input permutation has no pivots, declare the output of the marching to 
be null "0", and write n — > 0. Otherwise, consider a pivot (i, j) £ G(7t). Remove the 
hook emanating from (i, j), and move strictly to the northwest every diagram box in the 
rectangle with the corners (i, j), (I, m), into the only spaces available (i.e., by "hopping" 
over any hooks in the way). Do this by starting with the unique northwest box in the 
rectangle and continue left to right along the rows, and from top to bottom. It is easy to 
check that the resulting collection of boxes is necessarily the diagram of a permutation p. 

Let n— —>p denote that p is obtained from marching on D(7t) towards the pivot in row i. 

Example 1. Let n = 431 7625. We have D(7t) = {(1 , 1), (1 ,2), (1 ,3), (2, 1 ), (2,2), (4,2), (4,5), 
(4,6), (5,2), (5,5)}. The maximal corner is (5,5) and its pivots are the dots at (1,4), (2,3) 
and (3, 1 ). The boxes in the (3, 1 ), (5,5) rectangle of n are marked with Xs. Marching 
towards the pivot (2,3) we get p = 451 7326: 
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Note that two adjacent X's at (4,5) and (5,5) can become separated (to (2,3) and (4,3) 
respectively) after marching. 



Marching instead towards the pivot (3,1) we get p = 4357126: 



7t 



remove hook from 



















X 


X 








> X 


X 


□ l 


























I 





3,1) 



This time, some nonadjacent boxes in n become adjacent in p. 

More generally, suppose that 1 < ii < i 2 < . . . < i-k < I are a subset of the rows that 

contain pivots of (I, m). Consider n pi. Add a box to the diagram of pi at (I, pi(l)). 
This is the diagram of a new permutation p\, where the added box is the maximal corner 

for p\, and the dot in row i 2 is a pivot for this box. Now march p^ -^-> p 2 , and add a 
maximal corner similarly in row I to give p 2 . Repeat this process of marching and adding 

a box in row I until we obtain p = p k . We write n M ' X2 '-' lk > p to denote this more general 
K-marching operation; a total of k — 1 boxes are added. 

Example 2. Let n be as in the above example and suppose we march to the pivots (1,4) 
and (3, 1 ) of the maximal corner (5, 5), in succession. 
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p^: add a box at (5,4) p 

For any (3 G S n and any positive integer t, we define a rooted, labeled tree T t [fo) whose 
vertices are either labeled by or by a permutation (repetitions allowed). The root is 
labeled by (3. If a vertex is labeled by a permutation that has its last descent weakly 
smaller than t, or is labeled by 0, then declare that vertex to be a leaf. Otherwise, the 
children of a vertex are indexed by the output of all ways of marching from that vertex. 
One can check easily that in finitely many steps, this growth process terminates, giving 
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%[$). Note that %[$) is a pruning of %{$) for t < s. Define KT t (|3) similarly, using 
instead the K-marching operation (and similarly, K7t((3) C K7^(|3)). Finally, if a leaf vertex 
v is labeled n, call it a 7r-leaf . 

We will be particularly interested in the cases that KT S (|3) or T s (|3) has exactly one la- 
beled leaf (i.e. not by 0). The best-behaved cases are when |3 is "2143-avoiding," also 
known as "vexillary," in which case KT s [fi) has at most one leaf for any s. In the other 
direction, if n has a unique descent 7r(i) > n{i + 1 ), at i = s (called a Grassmannian 
permutation), N G N, and |3 is the N -stabilization of of n, meaning 

(3(i)=i fori<N, (3(1) = N + n{i - N) fori>N, 

then KT S ( (3 ) will have only one labeled leaf, and it will be labeled n. 

We are now ready to introduce the family of Schubert problems covered by our main 
theorem. There is a standard operation * n on two permutations cr, oc G S n . Let cr * n oc be 
the permutation in S2 n whose matrix is the direct sum of G(cr) and G(cx). For example, 
id * n oc is just the n-stabilization of oc. Let cr G S n be a permutation whose last descent 
is I, and let I < t < 2n be an integer. Suppose that oc G S n is such that K7^(id * n oc) 
contains a single leaf v with label (v) ^ 0; let label (v) = p. Under these circumstances, call 
(a, p, n) G x Soo a truncation Schubert problem subjugate to (t, oc). 

Theorem 1. If (cr, p, n) G x is a truncation Schubert problem subjugate to (t, cx) then 

(-1 )«M+«(pMMc£ p = the number ofn-leaves o/KT t (cr * n oc). 

In the cohomology case, we can also say 

C£ = the number ofn-leaves ofT t {o-k n oc). 

Example 3. Let cr = 3412 and oc = 3214 be permutations in S 4 , so cr * 4 oc = 34127658 G Sg. 
One can check that KT^id * 4 oc) has a single labeled leaf, labeled by the permutation 
12463578. Now K74(cr*4 oc) is given in Figure 1 below, and so by Theorem[TJ 

^X 3 4 12 ] ' [^X]2463578] [^X4 61 23578] [^X 3 6 14 257g] "I" [^X 3 5i 6247s] [^X 34 26157s] 

[^X46132578] [^X 36 15247s] [^X 36 24157g] [^X 35 26147s] [^X 3 625147«] 

where the expansion (|T]| has been done in the case Flags (C 8 ). 

As mentioned before the theorem, and spelled out in the corollary below, one family 
of truncation Schubert problems comes from Grassmannian permutations. In the coho- 
mology case, these were given a (different) positive combinatorial formula by M. Ko- 
gan IIKoUll . 

Corollary 1. Let u G S n have last descent at I and let p be a Grassmannian permutation with 
unique descent at t, where I < t < n. Then for any n G S^, 

(-1 )«(<r)+t(pMM C re p = the number ofn-leaves o/KT t (cr* n p). 

In the cohomology case (treated in |Ko01 [), 

C^ p = the number ofn-leaves ofT t {<j* n p). 

If we also assume that cr is grassmannian and moreover t = I, the first conclusion of 
Corollary [T] computes the K-theory Littlewood-Richardson coefficients of |Bu02a|, while 
the second conclusion computes the classical Littlewood-Richardson coefficients. 
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71 = 34127658 
FIGURE 1. The tree K7;(341 27657); see Example 3. 

Example 4. Let cr = 321 e S 3 , p = 132 and cr* 3 p = 321465. The tree K7^(321465) is given 
in Figure 2 below. Using this, the expansion Q for Flags (C 6 ) is 

[0X32,] • [CX, 32 ] = [CX421356] + [CX341256] - [0X 4 3,25 6 ]- 



There is an isomorphism of Flags (C n ) to itself induced by sending each vector subspace 
V to its orthogonal complement V -1 (with respect to an arbitrarily chosen bilinear form). 
The induced automorphism of K(Flags(C n )) gives the symmetry C" p = C™°™° WopWQ . 
This observation, combined with the corollary (or the theorem), gives, e.g., a subtraction- 
free formula also for the Schubert numbers p where p is Grassmannian of descent t 
which is weakly smaller than the^irsf descent of cr. 

Theorem [l] also handles some new (but apparently limited) cases of Schubert problems 
(a, p) where neither 0" nor p are Grassmannian permutations. This differs from other 
formulas, see, e.g., HES981 IBuU2al IBSY031 IKoUTl ILoTl ILeU5l ItSOl IMUTl I5%1 IV031 . 

Example 5. The tree K7/( 12345987610) has a single leaf indexed by a permutation, and 
that permutation is 12346985710. Hence a product of [(9 Xl 2345935710] with any [0 Xp ] where 
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71 = 321465 

FIGURE 2. The tree KT 2 (321465); see Example 4. 

p G S5 is covered by Theorem [TJ and in particular in K(Flags(C 10 ) ): 
[O 

X-12346985710-1 ' [^X41352] [^X4i 36298571 o] "F L^X41 35698271 (J [^X4i2 

is a nontrivial expansion which is not computed any previously known (subtraction-free) 
multiplication formula. We remark that Theorem [T] is the first to give a positive formula 
for even the cohomology expansion in H*(Flags(C 10 )): 

[Xl 234698571 o] ' [X41352] = [X41 36298571 ol + [X41 35698271 o] • 



1 36598271 0-1 



3. Proof of Theorem 1 and Corollary 1 



We begin by recalling A. Lascoux and M.-P. Schiitzenberger's Grothendieck polyno- 
mials |LS82], albeit via a rather unconventional definition. Let X = {x-| , X2, . . . } be a 
collection of commuting independent variables. To each n e Soo, there is an associated 
Grothendieck polynomial in the {xi}, and these polynomials satisfy the following crucial 
recursion: 

Theorem 2. (cf. |L0HILe03j ) For any permutation y e with last descent g, letvn>g be the 
largest integer such that y(m) < y(g) and sety' = yt gom . Suppose that 1 < U < i 2 < . . . < 
i s < g are the positions such that £(y'ti jMg ) = l{y') + 1. Then the (K-theory) transition formula 
of A. Lascoux |L01| (we give the formulation |Le03, Cor. 3.10]) holds: 

(2) <5 Y (X) = ©y (X) + (X B - 1 ) [&y (X) • (I - t ilf4g ) • • • (I - t isf4g )] , 

where acts on the {&i(X)} by <&z(X) ■ tj<_>t = <8fct,- m (X) and I acts as the identity operator. 
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This, and the base case <5id(X) = 1, uniquely determine the Grothendieck polynomials 
(the usual definition is via isobaric divided difference operators). Together, these polyno- 
mials form a Z-linear basis of Z[X] and satisfy 

a (X)0 p (X) = Y_ Ca,p<5n(X). 

For any positive integer t, define the truncation homomorphism r t : Z[X] — > Z[X] by 
r t (f(X))=f(x 1) ...,x t ,0,0,...). 

Theorem [T] is immediate from the second formula of the following result: 

Theorem 3. For any y G Soo, we have 

T t («5 Y (X)) = ^(-l) £(Y) - £(label(V %iabel(v)(X), 
v 

wZzere fZze sum is over all leaves v ofKT t {y) such that label (v) ^ 0. 

If cr G S n Ztas its Zflsf descent weakly smaller than t, and a G S n is arbitrary, then 

<9 ff (X)r t (<5 id * na (X)) = ^(-l) £(t7)+£(a) - £(labeltv)) C5 lab ei (v) (X), 

v 

wZzere f/ze sum is ouer aZZ Zeaues v o/K7^(o"* n a) such that label(v) ^ 0. 

Proof. To expand r t (0 Y (X)) we will need the following lemma, which connects the dia- 
gram moves of Section 2 to the (K-theory) transition formula in Theorem |2j It also gives 
an alternative form of a substitution formula of |LRS04|. 

Lemma 1. (cf. [LRS04 ) Under the assumptions of Theorem^we have that 

(i) (ii,y(ii)), . . • , (i s ,y(i s )) are thepivots ofthemaximal corner (g,y _1 (ra)) G D(y); 

(ii) the following formula holds: 

r g _ 1 (0 Y (X)) = Y_ (-D £(Y) ^ (T) r g -i(^T(X)) 

i 

y — >T 

where the summation ranges over all subsets X o/{ii , . . . , i s }. 

Proof. Observe that the diagram of y ' differs from the diagram of y only in that the maxi- 
mal accessible box of y has been removed (and thus there is a dot of G (y ') in that position 
instead). 1 Now, for any index 1 < a < g, £(y't aog ) = l[y') + 1 holds if and only if 
y'(a) < y'(g) and the rectangle defined by (a,y'(a)) and (g,y'(g)) contains no other dots 
of G(y'); that is, if and only if (a,y'(a)) is a pivot of the maximal accessible box of y. 
Hence (i) holds. 

Thus in view of (i), conclusion (ii) follows easily by expanding the K-transition formula 
from Theorem|2J observing that (—1 ) #J = (—1 )*(yM(t^ and setting x g = 0. □ 



1 This seems to be the main reason to work with diagrams of permutations rather than inversion sets 
(which more easily generalize to other root systems). 
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From the above lemma, we have 

Tg-^fX)) = ^(-D {M - {(T) T H ((5 T (X)) 

I 

y — >t 

(3) = ^(-1) £ M-'W(5 T + ^(-1)^HW V]( g T(X))) 

A B 

where A consists of those t appearing from marching from y that have last descent at g — 1 
or smaller, and B consists of those that still have their last descent at g. It is not hard to 
see that a finite number of iterations of marches from y results in or a permutation with 
last descent weakly smaller than g — 1 . Thus after repeated application of Lemma [l] on 
® we expand r g _i(0 T (X)) into the sum of Grothendieck polynomials indexed by such 
permutations (in particular we have just used the fact that if a permutation x has last 
descent t and has no pivots, then r t _i (0 T (X) ) = 0). Therefore, the first conclusion follows 
by iterating the operation of setting the variables x g _i , x g _ 2 , . . . , x t+1 to zero in succession. 

Only a little more is necessary for the second conclusion of the theorem. Using Theo- 
rem |2 and induction, it is easy to check that 

ff (X)0 id * nP (X) = ff * nP (X). 

Since the last descent of a is I, then only the variables Xi , . . . , X\ appear in <3 a (X). Hence 
because I < t, we have r t ((J5 a (X)) = S5 a (X) and so 

(4) <3 ff (X)r t ((5 id * na (X))=r t (<5^ 

We conclude by applying the first conclusion to y = cr* n <x and observing that l{a* n <x) = 
t{a)+t{ot). □ 

Proof of Corollary 1: Since r t (<3id* nP (X)) = <3 P (X) |FK94[, it follows from our above discus- 
sions that the hypotheses of Theorem [T]hold. □ 

Extensions to equivariant cohomology. The structure constants in the equivariant coho- 
mology ring Hj(Flags(C n ) ) are polynomials in a second set of variables {y i , y 2 , ...,}, and 
are known to have a positive expansion in {ut+i — yi} (as proven in IGrOll V It does not 
seem easy to extend our techniques to apply to this richer problem. While the transition 
formula does have an equivariant extension, and one can state an equivariant truncation 
formula, this formula involves the {yt} individually rather than as differences. In trying 
to group them into differences one leaves the realm of subtraction-free formulae. 
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